jit.profile and jit.allocprof: small hardenings and improvements - #663
jit.profile and jit.allocprof: small hardenings and improvements#663damiansirbu wants to merge 7 commits into
Conversation
|
Draft while I fold in additional hardening surfaced by large-modpack validation. |
|
A few more commits on top, all from running this as the backend of an in-game profiling UI on a full GAMMA install (600+ mods). Buffer sizing. Interval jitter. Heap-allocated tables. GC state getter, jit.util.gcstat. Test coverage. |
damiansirbu
changed the title
jit_profile_callback ran exit(EXIT_FAILURE) after a failed lua_pcall, so one error in the sampling callback killed the whole process. The engine panic handler (CScriptEngine::lua_panic) already logs the stack and returns, so the error stays visible. Drop the exit, clear the callback thread stack, and keep profiling. The failed sample is dropped.
luaJIT_profile_stop cleared ps->g before profile_timer_stop, so the Windows timer thread could enter profile_trigger, read the now-null g, and dereference it. Join the timer thread first, then clear the state. No trigger runs after the join.
…frames whole allocprof_record cut the stack key mid-frame at the buffer bound, so a truncated frame read as an unattributable stack and lost bytes to "unknown". Cut at the last ';' instead, so a partial frame drops whole. Raise the CPU dumpstack buffer 2048->8192 and the alloc stack key 512->4096 so a 64-deep capture holds the whole stack without mid-frame truncation.
The CPU sampler slept a fixed interval, so a mod running on a fixed schedule could phase-lock to it and be systematically over- or under-sampled. Draw each sleep from an exponential around the mean, the same way the allocation profiler already draws its sample distance, so the sample times are never periodic.
The leaf+stack aggregation tables were 80MB of static image data (4096+16384 slots x 4104B) after the key-size increase. Allocate them with calloc on the first jit.allocprof.start instead: a process that never profiles holds no table memory. The tables stay alive after stop because consumers dump after allocprof.stop(); reset and slot reads are NULL-guarded for calls before the first start.
The allocation profiler measures bytes but cannot show GC schedule health:
how close the collector is to its next step and how far behind it runs.
Those live in GCState (threshold, estimate, debt) and were not Lua-readable;
collectgarbage("count") exposes only total.
Add jit.util.gcstat() returning {total, threshold, estimate, debt} in bytes,
mirroring jit.util.traceinfo's table shape. Double-encoded, so an MSize past
2 GB does not wrap the int32 field. Read-only: it exposes engine state and
computes nothing, so it never changes VM behaviour. Manifest entry added.
The profiler can show GC time share but not how often the collector runs. Add a monotonic cycle counter incremented at both GCSpause end-transitions in gc_onestep (the single chokepoint every collection passes through, so it catches allocation-driven, parallel, and explicit collects alike), and return it as a fifth gcstat field. The counter is a file-scope global in lj_gc.c, not a GCState field, so it does not shift global_State layout or the save format.
|
Added a GC-cycle counter to
Mechanism: the counter increments at the two Lua usage: A tool samples Tested:
One caution for anyone adding a counter to the VM structs: an earlier build put the field in |
7064a8a to
f42f404
Compare
2 participants
Summary
Robustness improvements to the jit.profile / jit.allocprof sampler from #641.
The profile callback recovers on its own and sampling continues uninterrupted.
The stop path joins the sampler thread before clearing its state, keeping shutdown and the timer thread cleanly separated.
Callback recovery keeps the session running
The profile callback runs under lua_pcall in jit_profile_callback (lib_jit.c).
Upstream LuaJIT hands a non-zero pcall status to the panic handler and then calls exit(EXIT_FAILURE), which suits the luajit CLI but is heavy for a game.
In this fork the panic handler already logs and returns.
CScriptEngine::lua_panic (script_engine.cpp:179) prints the stack and output, then returns 0.
The improvement leans on that: it drops the exit, clears the callback thread stack with lua_settop(L2, 0) so nothing accumulates across samples, and sampling carries on.
This matters most at the LuaJIT 2.0.4 32-bit allocation ceiling during a long capture, where an out-of-memory raise in the callback is absorbed and the run continues.
Timer thread joined before its state is cleared
On stop, luaJIT_profile_stop (lj_profile.c) cleared ps->g = NULL and then stopped the timer.
The Windows sampler runs on its own thread, which reads ps->g inside profile_trigger and dereferences it (g->hookmask, lj_profile.c:163-167).
Joining that thread before clearing its state keeps the two from overlapping on ps->g.
The stop now joins the sampler thread first, setting the abort flag and waiting on it (WaitForSingleObject(ps->thread, INFINITE), lj_profile.c:287-290).
ps->g clears once the thread has exited, so the timer path always reads a live g.
Files changed
Testing
Built locally (DX11, 0 errors) and run on a live GAMMA session with both profilers compiled in.
For the callback improvement, a probe armed a callback that raised every 10ms for about 6 seconds, roughly 600 raises, then stopped it.
The game stayed alive throughout and kept logging after the probe cleared the callback.
The timer improvement follows from the source above: the deref is direct, and joining first keeps the clear ordered after the thread exits.
The sampler ran a full session with a clean stop.